Phantom-BTB: Improving Branch Target Buffer Performance by Leveraging the On-Chip Memory Hierarchy

Modern processors use Branch Target Buffers (BTB) to predict the target address of branches so that they can fetch ahead in the instruction stream increasing concurrency and performance. Ideally, BTBs would be large enough to capture the entire working set of the application and small enough for fast access and practical on-chip dedicated storage. Depending on the application, these requirements are at odds. For example, commercial applications that exhibit large instruction footprints benefit from large BTBs. This work introduces a new BTB design that accommodates large instruction footprints without dedicating expensive on-chip resources. In the proposed Phantom-BTB (PBTB) design, a conventional BTB is augmented with a virtual table that collects branch target information as the application runs. The virtual table does not have fixed dedicated storage. Instead, it resides in reserved physical address space and is transparently allocated in the on-chip caches except the L1, at cache line granularity. The entries present in the virtual table are proactively prefetched and installed in the dedicated conventional BTB, thus, expanding its perceived capacity. Experimental results with commercial workloads under full-system simulation demonstrate that PBTB improves performance over a 1K-entry BTB by 5.6% on average and up to 10.3%, while using only 89 bytes of extra storage. By adding a small prefetch buffer, performance with PBTB increases to 6.8% on average and to a high of 12.3%, while the storage overhead is only 8% of a conventional 1K-entry BTB. This performance matches the improvement of a conventional 4K-entry, one-cycle access BTB, while the dedicated storage is 3.6 times smaller.